University System of Maryland: Maryland Course Redesign Initiative

Frostburg State University

Course Title: General Psychology
Redesign Coordinator: Megan Bradley

Project Abstract
Final Report (as of 6/1/09)

Project Abstract

Frostburg State University (FSU) plans to redesign General Psychology, the first required course for psychology majors and also required for five other majors. In the General Education Program (GEP), this course is one of five that satisfies the social sciences requirement. Of these course options, General Psychology has the highest enrollment with approximately 67% of all students taking it. The average annual course enrollment is approximately 900 students with 50 students per section, and the (drop-failure-withdrawal (DWF) rate is 12%. On average, 12 sections are offered in the fall semester and six in the spring (occasionally a summer section is offered). This course is taught by both full-time faculty (50%) and adjuncts (50%).

The redesign of General Psychology will address several academic problems: staffing issues, course drift, redundancy of effort, and lack of active learning. First, fiscal constraints at the university have resulted in the department relying on an increasing number of adjuncts who may or may not receive proper guidance in course development. Additionally, in the past two years, the department has had to rely on master's level graduate students to teach the course as full instructors. Second, since General Psychology is not standardized, each instructor creates his or her own version of the course. Thus, multiple versions of General Psychology are taught by a variety of instructors at differing skill levels, resulting in "course drift." This drift may result in overall lower course quality. Third, instructor's use of time is inefficient and leaves little time to assist the student who may need extra help. Instructors spend a lot of time on redundant tasks, grading exams by hand and entering grades. Finally, the course’s lecture-based approach does not promote motivation, interest in course content, active learning, or account for different learning styles, which may contribute to the 12% DWF rate. It is the redesign team's goal to become more efficient and effective through the course redesign.

Using the Replacement Model, FSU will collapse three sections of 50 students into one section of 150 students and will reduce the number of in-class meetings by half. During the in-class meetings, interactive activities will be used for most of the period rather than lectures. In-class activities will be designed to promote active learning and to follow-up and coincide with course material for the week. Additionally, the in-class meetings will prepare the students for the next week's topic. For the remainder of the time, technology will be used for assessment and discussion of weekly topics. Via online assessment, students will receive immediate feedback on their performance and areas in need of improvement. Undergraduate learning assistants (ULAs) will help with peer tutoring, discussions, other learning activities assigned online, and technology issues that arise.

The redesigned course will enhance quality in several ways. The redesign of General Psychology will improve the standardization of the course, which will help prevent course drift. Further, the redesign will enhance quality by providing more learner-learner interaction, instructor-learner interactions, and active learner-content interaction. With the use of time-saving online assessment, instructors will have more time to interact with students, either in-class or in the online environment. Using online discussion forums, students will be divided into small groups where they can interact with each other, the instructor or the ULAs. The use of class time for activities rather than merely lectures will promote active learning and collaboration on small group projects. These types of interaction have the potential to promote student satisfaction with instruction, which will lead to increased motivation, interest, cohesiveness, and learning. Additionally, the use of online assessment will provide immediate feedback to the students and direct them to supplementary materials. Instructors and ULAs will evaluate student progress and feedback aimed at providing guidance to students who may need extra help. Students will have greater engagement with psychology content. Using the online class time, students will be linked to interesting content that is discussed by experts in the field of psychology and applicable to their lives.

The psychology department will assess the impact of course redesign in spring 2008 by comparing two redesigned sections of 100 students with two traditional sections of 50 students each. A common end-of-course comprehensive multiple-choice exam will be the main tool used for assessing the effectiveness of instruction. The test will contain a representative sampling of questions for common chapters covered and contain three types of questions: knowledge, critical thinking, and applications questions. All questions on this exam will be different from those found on previous quizzes and tests. However, the form of questions will be similar to ones previously used in that they will have similar prompts for directing students to access their knowledge, to think critically about the material covered, and to apply their knowledge. The redesign team will investigate training students to improve their abilities to answer critical thinking and application questions using practice and test questions often involving simulation of everyday and professional applications of psychological knowledge and thinking. In addition to the common final exam, the DWF rates will be analyzed. The same techniques will be used following full implementation of the redesign by comparing learning outcomes in the fully redesigned course with those collected from the traditional format in spring 2008.

The redesign of the General Psychology course will produce cost savings in several ways. Fewer instructors will be needed to teach the course since the number of sections will be reduced by a third, from 18 to six requiring as few as one full-time faculty member (vs. nine) and three adjuncts (vs. nine) to teach the course. This change will allow full-time faculty to teach upper-level courses. The cost-per-student will be reduced from $89 to $32, a 64% decrease. Interested full-time faculty will contribute their expertise to the redesign of the course, standardizing it and enhancing the quality. With improved consistency, adjuncts will be able to teach the course without concern for course drift and will receive training to help them with the teaching process. Since the department is not staffed at full capacity, the cost savings from the redesigned course may be used to fund new full-time faculty.

Final Report (as of 6/1/09)

Impact on Students

In the redesign, did students learn more, less or the same compared to the traditional format?

Improved Learning

During the spring 2008 pilot, four sections of General Psychology were offered: two traditional sections taught by seasoned adjunct faculty members and two redesigned sections taught by one full-time and one adjunct faculty member. Both were primary members of the redesign team. A comprehensive, required final exam of 50 multiple-choice questions was created by the chair of the redesign team (who did not teach any sections). Instructors were blind to its content.

Results indicated students in the redesign sections (Mean = 75%) performed significantly better than students in the traditional sections (Mean = 68%). Dividing the exam into factual versus conceptual questions yielded similar results. Students from the redesign performed significantly better than students from traditional sections. Follow-up analyses revealed that final exam scores were positively correlated with student scores on the mastery quizzes (r = .523). Thus, this aspect of the redesign appeared helpful for students.

An optional extra-credit essay that asked students to write about prejudice was also given with the final exam. Instructors were again blind to its content. A grading rubric provided points for each correctly used psychological concept in order to separate "general public" answers from answers by knowledgeable psychology students. Results indicated students in the redesign sections (Mean = 2.845) performed significantly better than students in the traditional sections (Mean = 1.092). Follow-up analyses revealed that students' essay scores significantly correlated with both their grades on the semester-long prejudice project (r = .328) and their grades for online discussions (r = .244). Thus, these online activities assisted student acquisition of knowledge.

During the fall 2008 full implementation, the team compared performance on 43 questions out of the original 50 questions from the pilot semester final to both the traditional and redesigned sections from spring 2008. Students from full redesign (Mean = 77%) performed significantly better than students from pilot redesign (Mean = 70%), who performed significantly better than students from traditional sections (Mean = 65%). The effect size was a strong .825. The same pattern of results (full redesign > pilot redesign > traditional sections) was obtained comparing factual versus conceptual questions.

Improved Retention

The average DFW rate for General Psychology was 12.5% during 1998 - 2006. The DFW rate has been increasing in recent years, reaching a high of 18% the semester prior to the pilot. The fall 2008 full implementation rate was 12.8%, similar to the rate prior to redesign.

Impact on Cost Savings

Were costs reduced as planned?

FSU saved more than they anticipated. As planned, the team reduced the number of instructors (full-time and adjunct) needed to teach the course. The cost-per-student was reduced from $89 to $26 (not $32 as first estimated), a 71% decrease.

The additional cost savings were due to FSU’s not paying as many undergraduate learning assistants (ULAs) as originally planned. The team planned to pay 12 ULAs. In the pilot, they were not paid. Instead, FSU developed a ULA certification process, where students registered for credit and were not paid. As they did during full implementation, FSU in the future will pay half the ULAs (who work in the lab) $6,000 total per year; the other half (who do other activities) will be part of the certification process and will not be paid.

Lessons Learned

Pedagogical Improvement Techniques

What techniques contributed most to improving the quality of student learning?

  • Changes made from pilot to full implementation. Despite the successes achieved in the pilot semester, students from the redesigned sections often did not complete online assignments. This lack of completion reached such high levels that the team referred to it as "deadline disorder." Despite twice-weekly reminders from ULAs, students had difficulty scheduling their time to complete tasks. The blended/hybrid aspect of the course proved to be too flexible for students who reported needing more structure.

Consequently, a number of adjustments for the fall 2008 full implementation were made. The number of required chapters and online assignments were reduced by eliminating the pre-lecture quizzes and self-assessments. The time frame to accomplish the remaining online activities was extended from one to two weeks. A required computer lab run by ULAs was implemented. In addition, small-group or individualized assistance was made available using university resources. Finally, the team broke up the semester into three modules, each ending with an in-class exam instead of offering one comprehensive final exam. This modular approach allowed students to learn new content within a tighter time frame, be tested on that learning and move on to new material, following the learning principle of the "spacing effect."

  • Using a computer lab and ULAs for online activities. In the pilot semester, computer labs were not used. Students met with the instructor only once per week for 75 minutes. Students were expected to complete the online portion of the course on their own. The team discovered that students did not meet deadlines despite twice-weekly reminders. Computer labs run by ULAs were first used in the fall 2008 full implementation semester to help students complete work successfully. Students earned attendance points for lab. In the spring 2009 semester, ULAs delivered brief PowerPoint presentations describing the online work to be completed for the week, and they described the grading rubric for the activities. After the presentation, ULAs were available to help students with their work. After implementing the required lab, students were more likely to meet deadlines.
  • Low-stakes quizzing. In the pilot semester, two types of quizzing were used: pre-lecture quizzes (PLQ) and mastery quizzing (MQ). The PLQs were used to guide the instructor in preparing face-to-face material and to provide students with a computer-generated study plan. If students were struggling with a specific content area (below 75%), the instructor would emphasis this content in class. Students were expected to complete these quizzes prior to the face-to-face session. When instructors evaluated the PLQ reports that measured “difficult content areas for students,” they found that students often scored below 75% in all content areas, thus defeating the purpose of the PLQs. Additionally, students thought the PLQs were too much work since they also were assigned mastery quizzes during the week.

MQs were provided for each chapter covered in the course to help students prepare for the comprehensive exam. In the pilot, the due date for the MQs was the date of the final exam. Students often waited until the final exam week to complete the MQs rather than doing so gradually throughout the semester. In the fall 2008 full implementation, the team set due dates for MQs every four to five weeks.

  • Online discussion. Students participated in three online discussions per semester lasting two weeks in duration, which accounted for 30% of their grade. Students responded to three questions in week one and to their peers in week two. Discussion activities promoted social interaction and provided an opportunity for students to apply course content to their lives. They also allowed students to demonstrate knowledge in a way other than by taking mastery quizzes. The discussions were graded by ULAs who were trained in using a rubric to score student responses. The lab assistants reviewed the rubric and described the discussion activity with students during lab and offered assistance to students when completing the activity.
  • Semester-long prejudice perspective. Students provided online written responses using the textbook or outside resources to questions that related to prejudice. This activity, which accounted for approximately 30% of the student’s grade, occurred three times per semester and was two weeks in duration. As with discussion activities, this activity provided an opportunity for students to demonstrate their understanding of specific concepts in ways other than objective testing.
  • Adding PowerPoints to structure active learning activities. In the pilot semester, the face-to-face sessions emphasized active learning and lectures and avoided direct instruction. In fall 2008, brief direct instruction was added. The redesign team developed PowerPoint presentations that integrated active-learning activities. These presentations served as a guide for the instructor and provided needed structure for the students. Using the presentations, the instructors were better able to provide necessary content to students to facilitate their engaging in meaningful activities.
  • Supplemental instruction (SI) and tutoring. SI and tutoring were implemented in fall 2008 to provide extra out-of-class assistance for students. Students attending SI and tutoring sessions could earn up to three points per week as extra credit. SI was coordinated through FSU’s Student Support Services, and tutoring was coordinated through FSU’s Tutoring Center . Trained ULAs served in these positions. SI was offered in a group setting, and tutoring was offered as a one-to-one service. SI sessions were initially offered two times per week, but an additional session was offered based on demand (an average of 20 students per session attended). Students liked the opportunity to discuss course content in this informal setting and to learn from peers.

Cost Reduction Techniques

What techniques contributed most to reducing costs?

  • Shared resources. All instructors in all sections of the course used a common syllabus, which eliminated syllabus development by all instructors. With the standardized course, it was possible to develop PowerPoint presentations that integrated weekly activities for each topic. In this way, fewer instructor hours were needed to develop these activities.
  • Consolidating sections. Standardizing the course allowed the team to increase the number of students per section from 50 to 150 and to offer fewer sections. The course went from 18 sections per year to six, and consequently the number of instructors needed to teach the course, both full-time and adjunct, was reduced.
  • Reduced textbook costs. The least expensive option for students who did not want to purchase a hard copy of the textbook was to use the e-book, which was available within the online portion of the course (PsychPortal). When students purchased access to the course through the FSU bookstore or through the PsychPortal web site, they also accessed the e-book. The e-book was generally more than half the cost of a new textbook. Additionally, if students dropped the course before completing it, they had access to the online portion of the course and the e-book during the next semester for free.

Using one textbook for all sections meant better resale value for students. Previously seven textbooks were used. Using only one increased the number of used textbooks for students to purchase, which were lower in cost than new books.

Implementation Issues

What implementation issues were most important?

  • Departmental buy-in. Before starting the redesign, the psychology department discussed the redesign process and committed to it. The support of the department throughout the process freed the team to focus on the redesign efforts.
  • Team approach. The three-person team was unique in that it consisted of a faculty member and an adjunct faculty member, both of whom routinely taught the course, and a faculty member who did not teach the course. All had considerable experience with instructional technology. Given the various strengths of the team members, duties were split among the group.
  • Instructor buy-in. Since several adjunct instructors will teach the redesigned course, the team had to sell the new design to this group. No major difficulties were experienced during full implementation.
  • Publisher materials. Worth Publishers provided a wide selection of resources that were useful in the classroom. These resources allowed the team to focus on the redesign instead of developing classroom activities. PsychPortal, the primary online web site, was an excellent resource. The site provided personalized support and reports that allowed the team to track student activity. It also avoided placing a large demand on the FSU servers.
  • Undergraduate learning assistants (ULAs). A major component of the team’s redesign efforts was the implementation of an advanced student support system. The ULAs supervised small-group online activities (e.g., discussions, semester-long project) and generally assisted instructors in teaching the course. To support these efforts, the team developed a psychology leadership certificate (a specialized focus) and a new course to support the program. Two of the original ULAs from the pilot semester earned prestigious summer assistantships at major universities due, in part, to their experiences as peer leaders. A spring 2009 ULA earned a summer internship with the Sloop Leadership Institute. Most of FSU’s graduating ULAs have been accepted at graduate programs and have received or are in competition for assistantships. In addition, a paid internship opportunity was made available within the department created by one of the ULAs. This opportunity included work as a peer leader and also involved the ULA in the research/assessment component of the course.
  • Computer lab. In the pilot implementation the team identified a major problem with students not meeting online deadlines, which they termed “deadline disorder.” In response, the team decided to add a required computer lab where students worked on their online assignments. Although this has worked well, it has created additional issues that the team must be dealt with: scheduling lab rooms since the university does not have a lab large enough to accommodate the psychology course class size as well as the need for laptops.

Sustainability

Will the redesign be sustained now that the MCRI project is over ?

The support for the redesigned course has not wavered within the department, and there is every indication that it will remain. Since the redesign has been fully implemented, the team has moved to a maintenance role. At some point, some or all of the members of the original redesign team may need to be replaced by other members of the department. This issue is just beginning to be explored.

As with all courses, the redesigned course will continue to evolve and need updating with new book editions, materials, etc. The original team members were compensated for their time and effort through a combined grant from USM and the university. Since the grant process is now complete, the issue is whether new forms of compensation will become available to support continued efforts.

One of the major changes the team made from the pilot implementation to the full implementation was adding a course lab. Although it has been effective in increasing learning, the lab has added tasks (e.g., identifying and recruiting lab assistants, weekly monitoring of assistants and interns.) During the redesign process, the coordinator had release time that allowed her to complete some of these tasks. The team hopes that the university will support compensation options for future coordinators.

 

 

Quick Links:

University System of Maryland Main Page...